Some Objects Are More Equal Than Others: Measuring and Predicting Importance

We observe that everyday images contain dozens of objects, and that humans, in describing these images, give different priority to these objects. We argue that a goal of visual recognition is, therefore, not only to detect and classify objects but also to associate with each a level of priority which we call 'importance'. We propose a definition of importance and show how this may be estimated reliably from data harvested from human observers. We conclude by showing that a first-order estimate of importance may be computed from a number of simple image region measurements and does not require access to image meaning

Mulsemedia: State of the art, perspectives, and challenges

Mulsemedia-multiple sensorial media-captures a wide variety of research efforts and applications. This article presents a historic perspective on mulsemedia work and reviews current developments in the area. These take place across the traditional multimedia spectrum-from virtual reality applications to computer games-as well as efforts in the arts, gastronomy, and therapy, to mention a few. We also describe standardization efforts, via the MPEG-V standard, and identify future developments and exciting challenges the community needs to overcome

Novel colours and the content of experience

I propose a counterexample to naturalistic representational theories of phenomenal character. The counterexample is generated by experiences of novel colours reported by Crane and Piantanida. I consider various replies that a representationalist might make, including whether novel colours could be possible colours of objects and whether one can account for novel colours as one would account for binary colours or colour mixtures. I argue that none of these strategies is successful and therefore that one cannot fully explain the nature of the phenomenal character of perceptual experiences using a naturalistic conception of representation

Eye movements during scene inspection: A test of the saliency map hypothesis

What attracts attention when we inspect a scene? Two experiments recorded eye movements while viewers inspected pictures of natural office scenes in which two objects of interest were placed. One object had low contour density and uniform colouring (a piece of fruit), relative to another that was visually complex (for example, coffee mugs and commercial packages). In each picture the visually complex object had the highest visual saliency according to the Itti and Koch algorithm. Two experiments modified the task while the pictures were inspected, to determine whether visual saliency is invariably dominant in determining the pattern of fixations, or whether the purpose of inspection can provide a cognitive override that renders saliency secondary. In the first experiment viewers inspected the scene in preparation for a memory task, and the more complex objects were potent in attracting early fixations, in support of a saliency map model of scene inspection. In the second experiment viewers were set the task of detecting the presence of a low saliency target, and the effect of a high saliency distractor was negligible, supporting a model in which the saliency map can be built with cognitive influences that override low-level visual features

Ambiguous figures and the content of experience

Representationalism is the position that the phenomenal character of an experience is either identical with, or supervenes on, the content of that experience. Many representationalists hold that the relevant content of experience is nonconceptual. I propose a counterexample to this form of representationalism that arises from the phenomenon of Gestalt switching, which occurs when viewing ambiguous figures. First, I argue that one does not need to appeal to the conceptual content of experience or to judgements to account for Gestalt switching. I then argue that experiences of certain ambiguous figures are problematic because they have different phenomenal characters but that no difference in the nonconceptual content of these experiences can be identified. I consider three solutions to this problem that have been proposed by both philosophers and psychologists and conclude that none can account for all the ambiguous figures that pose the problem. I conclude that the onus is on representationalists to specify the relevant difference in content or to abandon their position

On Multifractal Structure in Non-Representational Art

Multifractal analysis techniques are applied to patterns in several abstract expressionist artworks, paintined by various artists. The analysis is carried out on two distinct types of structures: the physical patterns formed by a specific color (``blobs''), as well as patterns formed by the luminance gradient between adjacent colors (``edges''). It is found that the analysis method applied to ``blobs'' cannot distinguish between artists of the same movement, yielding a multifractal spectrum of dimensions between about 1.5-1.8. The method can distinguish between different types of images, however, as demonstrated by studying a radically different type of art. The data suggests that the ``edge'' method can distinguish between artists in the same movement, and is proposed to represent a toy model of visual discrimination. A ``fractal reconstruction'' analysis technique is also applied to the images, in order to determine whether or not a specific signature can be extracted which might serve as a type of fingerprint for the movement. However, these results are vague and no direct conclusions may be drawn.Comment: 53 pp LaTeX, 10 figures (ps/eps

Modelling search for people in 900 scenes: A combined source model of eye guidance

How predictable are human eye movements during search in real world scenes? We recorded 14 observers’ eye movements as they performed a search task (person detection) in 912 outdoor scenes. Observers were highly consistent in the regions fixated during search, even when the target was absent from the scene. These eye movements were used to evaluate computational models of search guidance from three sources: Saliency, target features, and scene context. Each of these models independently outperformed a cross-image control in predicting human fixations. Models that combined sources of guidance ultimately predicted 94% of human agreement, with the scene context component providing the most explanatory power. None of the models, however, could reach the precision and fidelity of an attentional map defined by human fixations. This work puts forth a benchmark for computational models of search in real world scenes. Further improvements in modelling should capture mechanisms underlying the selectivity of observers’ fixations during search.National Eye Institute (Integrative Training Program in Vision grant T32 EY013935)Massachusetts Institute of Technology (Singleton Graduate Research Fellowship)National Science Foundation (U.S.) (Graduate Research Fellowship)National Science Foundation (U.S.) (CAREER Award (0546262))National Science Foundation (U.S.) (NSF contract (0705677))National Science Foundation (U.S.) (Career Award (0747120)

Usability of the SAFEWAY@SCHOOL system in children with cognitive disabilities

PurposeSAFEWAY2SCHOOL is a programme based on several systems for the enhancement of school transportation safety for children. The aim of the study was to explore whether children with cognitive disabilities will notice, realise, understand, trust and accept the SAFEWAY2SCHOOL system and act in accordance with its instructions. Methods Fourteen children with cognitive disabilities and a control group of 23 children were shown five videos of scenarios involving journeys to and from school. During the first viewing visual scanning patterns were recorded with an eye tracking device. After a second viewing the participant was asked ten questions per scenario. Five questions addressed what the children saw on the video, and the remaining five what they would need to know and/or do within the scenario. Additional ratings of trust, likability, acceptability and usability were also collected. Results Very few differences were found in the visual scanning patterns of children with disabilities compared to children who participated in the control group. Of the 50 questions regarding what children saw or needed to know and/or do, only one significant difference between groups was found. No significant differences were found regarding self-reported ratings of trust, acceptability or usability of the system. Despite some significant differences across five of the 11 likability aspects, ratings were consistently high for both groups. Conclusions Children with cognitive disabilities proved that the SAFEWAY2SCHOOL system is as useful for them as it was for children in the control group. However, a valid estimation of the full utility of SAFEWAY2SCHOOL requires in situ testing of the system with these children